Aiming at the problem of single image degradation and high time complexity of exiting defogging methods under foggy weather, a fast visual optimization defogging algorithm based on atmospheric physical model was proposed. The proposed method firstly used threshold segmentation to find the sky region, and combined with binary tree algorithm to locate global atmospheric light precisely, and then adopted improved constrained least squares filter which can keep the edge detail and reduce noise to optimize original transmittance map. Finally, the fog image could be restored by atmospheric physical model, and the average gradient, information entropy and the visual information fidelity index were adopted to evaluate the image. The experimental results show that compared with the adaptive image enhancement method based on multi-scale Retinex algorithm, the image restoration based on independent component analysis, a quick visual image restoration method and the dark-channel prior de-hazing algorithm, the proposed method has good visual evaluation indexes and strong real-time processing capability.